<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  

  
  <title>shouzimu</title>
  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
  <meta property="og:type" content="website">
<meta property="og:title" content="shouzimu">
<meta property="og:url" content="https://shouzimu.gitee.io/index.html">
<meta property="og:site_name" content="shouzimu">
<meta property="og:locale" content="zh_CN">
<meta property="article:author" content="shouzimu">
<meta name="twitter:card" content="summary">
  
    <link rel="alternate" href="/atom.xml" title="shouzimu" type="application/atom+xml">
  
  
    <link rel="icon" href="/favicon.png">
  
  
    <link href="//fonts.googleapis.com/css?family=Source+Code+Pro" rel="stylesheet" type="text/css">
  
  
<link rel="stylesheet" href="/css/style.css">

<meta name="generator" content="Hexo 5.2.0"></head>

<body>
  <div id="container">
    <div id="wrap">
      <header id="header">
  <div id="banner"></div>
  <div id="header-outer" class="outer">
    <div id="header-title" class="inner">
      <h1 id="logo-wrap">
        <a href="/" id="logo">shouzimu</a>
      </h1>
      
    </div>
    <div id="header-inner" class="inner">
      <nav id="main-nav">
        <a id="main-nav-toggle" class="nav-icon"></a>
        
          <a class="main-nav-link" href="/">Home</a>
        
          <a class="main-nav-link" href="/archives">Archives</a>
        
      </nav>
      <nav id="sub-nav">
        
          <a id="nav-rss-link" class="nav-icon" href="/atom.xml" title="RSS Feed"></a>
        
        <a id="nav-search-btn" class="nav-icon" title="Search"></a>
      </nav>
      <div id="search-form-wrap">
        <form action="//google.com/search" method="get" accept-charset="UTF-8" class="search-form"><input type="search" name="q" class="search-form-input" placeholder="Search"><button type="submit" class="search-form-submit">&#xF002;</button><input type="hidden" name="sitesearch" value="https://shouzimu.gitee.io"></form>
      </div>
    </div>
  </div>
</header>
      <div class="outer">
        <section id="main">
  
    <article id="post-essync" class="article article-type-post" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2020/11/12/essync/" class="article-date">
  <time datetime="2020-11-12T12:07:45.000Z" itemprop="datePublished">2020-11-12</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 itemprop="name">
      <a class="article-title" href="/2020/11/12/essync/">es+canal+logstash优化用户查询</a>
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <p>记录一次使用es优化用户模糊查询的过程</p>
<p>相关中间件及版本如下</p>
<table>
<thead>
<tr>
<th>应用</th>
<th align="left">版本</th>
</tr>
</thead>
<tbody><tr>
<td>ElasticSearch</td>
<td align="left">5.5</td>
</tr>
<tr>
<td>mysql</td>
<td align="left">5.6</td>
</tr>
<tr>
<td>logstash</td>
<td align="left">-</td>
</tr>
<tr>
<td>canal</td>
<td align="left">1.1.4</td>
</tr>
<tr>
<td>Springboot</td>
<td align="left">2.0.5.RELEASE</td>
</tr>
</tbody></table>
<p>替换前：mysql模糊查询用户，耗时3s+</p>
<p>替换后：es查询id，再in查询数据库 100ms</p>
<h4 id="思路："><a href="#思路：" class="headerlink" title="思路："></a>思路：</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">用户表的更新走binlog同步，中间间使用canal和otter，二者介绍这里不再赘述</span><br></pre></td></tr></table></figure>



<h4 id="一、ElasticSearch配置"><a href="#一、ElasticSearch配置" class="headerlink" title="一、ElasticSearch配置"></a>一、ElasticSearch配置</h4><p>创建索引，5.5版本还需要设置type，比较蛋疼</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">1、创建索引member_info</span><br><span class="line">PUT &#x2F;member_info</span><br><span class="line"></span><br><span class="line">2、设置type member</span><br><span class="line">PUT &#x2F;member_info&#x2F;_mapping&#x2F;member</span><br><span class="line">&#123;</span><br><span class="line">	&quot;properties&quot;: &#123;</span><br><span class="line">	&#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">es工作到此结束</span><br></pre></td></tr></table></figure>



<h4 id="二、canal配置"><a href="#二、canal配置" class="headerlink" title="二、canal配置"></a>二、canal配置</h4><h5 id="1、canal-server端配置"><a href="#1、canal-server端配置" class="headerlink" title="1、canal server端配置"></a>1、canal server端配置</h5><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">直接使用的单机版</span><br><span class="line"></span><br><span class="line">github下载的canal.deployer-1.1.4.tar.gz</span><br><span class="line"></span><br><span class="line">解压，编辑 conf/example/instance.properties 修改以下关键配置</span><br><span class="line">canal.instance.master.address=数据库ip:数据库端口</span><br><span class="line">canal.instance.dbUsername=数据库用户名</span><br><span class="line">canal.instance.dbPassword=数据库密码</span><br><span class="line">canal.instance.filter.regex=member_info (注：这个地方我只需要监听这一个表的数据变化)</span><br><span class="line"></span><br><span class="line">启动</span><br><span class="line">bin/startup.sh</span><br></pre></td></tr></table></figure>


<h5 id="2、canal-客户端-otter的配置，这个地方写的代码，直接上"><a href="#2、canal-客户端-otter的配置，这个地方写的代码，直接上" class="headerlink" title="2、canal 客户端 otter的配置，这个地方写的代码，直接上"></a>2、canal 客户端 otter的配置，这个地方写的代码，直接上</h5><p>MemberCanalClient.java</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br><span class="line">170</span><br><span class="line">171</span><br><span class="line">172</span><br><span class="line">173</span><br><span class="line">174</span><br><span class="line">175</span><br><span class="line">176</span><br><span class="line">177</span><br><span class="line">178</span><br><span class="line">179</span><br><span class="line">180</span><br><span class="line">181</span><br><span class="line">182</span><br><span class="line">183</span><br><span class="line">184</span><br><span class="line">185</span><br><span class="line">186</span><br><span class="line">187</span><br><span class="line">188</span><br><span class="line">189</span><br><span class="line">190</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">@Component</span></span><br><span class="line"><span class="meta">@Slf4j</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">MemberCanalClient</span> </span>&#123;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">private</span> TransportClient transportClient;</span><br><span class="line">    </span><br><span class="line">    String index = <span class="string">&quot;member_info&quot;</span>;</span><br><span class="line">    String type = <span class="string">&quot;member&quot;</span>;</span><br><span class="line">    </span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">initElasticSearch</span><span class="params">()</span> <span class="keyword">throws</span> UnknownHostException </span>&#123;</span><br><span class="line">        Settings settings = Settings.builder().put(<span class="string">&quot;cluster.name&quot;</span>, <span class="string">&quot;my-application-1&quot;</span>).put(<span class="string">&quot;xpack.security.user&quot;</span>, <span class="string">&quot;用户名:密码&quot;</span>).put(<span class="string">&quot;client.transport.sniff&quot;</span>, <span class="keyword">false</span>).build();</span><br><span class="line">        transportClient = <span class="keyword">new</span> PreBuiltXPackTransportClient(settings);</span><br><span class="line">        TransportAddress transportAddress = <span class="keyword">new</span> InetSocketTransportAddress(InetAddress.getByName(<span class="string">&quot;ip&quot;</span>), <span class="string">&quot;端口&quot;</span>);</span><br><span class="line">        transportClient.addTransportAddress(transportAddress);</span><br><span class="line">    &#125;</span><br><span class="line">    </span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">otterSync</span><span class="params">()</span> </span>&#123;</span><br><span class="line">        <span class="comment">// 创建链接</span></span><br><span class="line">        CanalConnector connector = CanalConnectors.newSingleConnector(<span class="keyword">new</span> InetSocketAddress(AddressUtils.getHostIp(),</span><br><span class="line">                <span class="number">11113</span>), <span class="string">&quot;example&quot;</span>, <span class="string">&quot;&quot;</span>, <span class="string">&quot;&quot;</span>);</span><br><span class="line">        <span class="keyword">int</span> batchSize = <span class="number">1000</span>;</span><br><span class="line">        <span class="keyword">int</span> emptyCount = <span class="number">0</span>;</span><br><span class="line">        <span class="keyword">try</span> &#123;</span><br><span class="line">            connector.connect();</span><br><span class="line">            connector.subscribe(<span class="string">&quot;member.member_info&quot;</span>);</span><br><span class="line">            connector.rollback();</span><br><span class="line">            <span class="keyword">while</span> (<span class="keyword">true</span>) &#123;</span><br><span class="line">                <span class="comment">// 获取指定数量的数据</span></span><br><span class="line">                Message message = connector.getWithoutAck(batchSize);</span><br><span class="line">                <span class="keyword">long</span> batchId = message.getId();</span><br><span class="line">                <span class="keyword">int</span> size = message.getEntries().size();</span><br><span class="line">                <span class="keyword">if</span> (batchId == -<span class="number">1</span> || size == <span class="number">0</span>) &#123;</span><br><span class="line">                    emptyCount++;</span><br><span class="line">                    System.out.println(<span class="string">&quot;empty count : &quot;</span> + emptyCount);</span><br><span class="line">                    <span class="keyword">try</span> &#123;</span><br><span class="line">                        Thread.sleep(<span class="number">1000</span>);</span><br><span class="line">                    &#125; <span class="keyword">catch</span> (InterruptedException e) &#123;</span><br><span class="line">                    &#125;</span><br><span class="line">                &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">                    emptyCount = <span class="number">0</span>;</span><br><span class="line">                    <span class="comment">// System.out.printf(&quot;message[batchId=%s,size=%s] \n&quot;, batchId, size);</span></span><br><span class="line">                    printEntry(message.getEntries());</span><br><span class="line">                &#125;</span><br><span class="line">    </span><br><span class="line">                connector.ack(batchId); <span class="comment">// 提交确认</span></span><br><span class="line">                <span class="comment">// connector.rollback(batchId); // 处理失败, 回滚数据</span></span><br><span class="line">            &#125;</span><br><span class="line">        &#125; <span class="keyword">finally</span> &#123;</span><br><span class="line">            connector.disconnect();</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    </span><br><span class="line">    <span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">printEntry</span><span class="params">(List&lt;Entry&gt; entrys)</span> </span>&#123;</span><br><span class="line">        <span class="keyword">for</span> (Entry entry : entrys) &#123;</span><br><span class="line">            <span class="keyword">if</span> (entry.getEntryType() == EntryType.TRANSACTIONBEGIN || entry.getEntryType() == EntryType.TRANSACTIONEND) &#123;</span><br><span class="line">                <span class="keyword">continue</span>;</span><br><span class="line">            &#125;</span><br><span class="line">    </span><br><span class="line">            RowChange rowChage = <span class="keyword">null</span>;</span><br><span class="line">            <span class="keyword">try</span> &#123;</span><br><span class="line">                rowChage = RowChange.parseFrom(entry.getStoreValue());</span><br><span class="line">            &#125; <span class="keyword">catch</span> (Exception e) &#123;</span><br><span class="line">                <span class="keyword">throw</span> <span class="keyword">new</span> RuntimeException(<span class="string">&quot;ERROR ## parser of eromanga-event has an error , data:&quot;</span> + entry.toString(),</span><br><span class="line">                        e);</span><br><span class="line">            &#125;</span><br><span class="line">    </span><br><span class="line">            EventType eventType = rowChage.getEventType();</span><br><span class="line">            System.out.println(String.format(<span class="string">&quot;================&amp;gt; binlog[%s:%s] , name[%s,%s] , eventType : %s&quot;</span>,</span><br><span class="line">                    entry.getHeader().getLogfileName(), entry.getHeader().getLogfileOffset(),</span><br><span class="line">                    entry.getHeader().getSchemaName(), entry.getHeader().getTableName(),</span><br><span class="line">                    eventType));</span><br><span class="line">    </span><br><span class="line">            <span class="keyword">for</span> (RowData rowData : rowChage.getRowDatasList()) &#123;</span><br><span class="line">                <span class="keyword">try</span> &#123;</span><br><span class="line">                    operator(rowData, eventType);</span><br><span class="line">                &#125;<span class="keyword">catch</span> (Exception e)&#123;</span><br><span class="line">    </span><br><span class="line">                &#125;</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    </span><br><span class="line">    <span class="function"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">printColumn</span><span class="params">(List&lt;Column&gt; columns)</span> </span>&#123;</span><br><span class="line">        <span class="keyword">for</span> (Column column : columns) &#123;</span><br><span class="line">            System.out.println(column.getName() + <span class="string">&quot; : &quot;</span> + column.getValue() + <span class="string">&quot;    update=&quot;</span> + column.getUpdated());</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">operator</span><span class="params">(RowData rowData, EventType eventType)</span> </span>&#123;</span><br><span class="line">        System.out.println(eventType);</span><br><span class="line">        List&lt;Column&gt; columnList = rowData.getAfterColumnsList();</span><br><span class="line">        <span class="keyword">if</span> (eventType == EventType.DELETE) &#123;</span><br><span class="line">            columnList = rowData.getBeforeColumnsList();</span><br><span class="line">        &#125;</span><br><span class="line">        Member m = <span class="keyword">new</span> Member();</span><br><span class="line">        <span class="keyword">boolean</span> update = <span class="keyword">false</span>;</span><br><span class="line">        <span class="keyword">for</span> (Column column : columnList) &#123;</span><br><span class="line">            System.out.println(column.getName() + <span class="string">&quot; : &quot;</span> + column.getValue() + <span class="string">&quot;    update=&quot;</span> + column.getUpdated());</span><br><span class="line">    </span><br><span class="line">            <span class="keyword">switch</span> (column.getName()) &#123;</span><br><span class="line">                <span class="keyword">case</span> <span class="string">&quot;id&quot;</span>:</span><br><span class="line">                    m.setId(column.getValue());</span><br><span class="line">                    update = checkUpdate(update, column.getUpdated());</span><br><span class="line">                    <span class="keyword">break</span>;</span><br><span class="line">                <span class="keyword">case</span> <span class="string">&quot;mobile&quot;</span>:</span><br><span class="line">                    m.setMobile(column.getValue());</span><br><span class="line">                    update = checkUpdate(update, column.getUpdated());</span><br><span class="line">                    <span class="keyword">break</span>;</span><br><span class="line">                <span class="keyword">case</span> <span class="string">&quot;name&quot;</span>:</span><br><span class="line">                    m.setName(column.getValue());</span><br><span class="line">                    update = checkUpdate(update, column.getUpdated());</span><br><span class="line">                    <span class="keyword">break</span>;</span><br><span class="line">                <span class="keyword">case</span> <span class="string">&quot;pinyin_code&quot;</span>:</span><br><span class="line">                    m.setPy(column.getValue());</span><br><span class="line">                    update = checkUpdate(update, column.getUpdated());</span><br><span class="line">                    <span class="keyword">break</span>;</span><br><span class="line">                <span class="keyword">case</span> <span class="string">&quot;is_deleted&quot;</span>:</span><br><span class="line">                    m.setIsDeleted(column.getValue());</span><br><span class="line">                    update = checkUpdate(update, column.getUpdated());</span><br><span class="line">                    <span class="keyword">break</span>;</span><br><span class="line">                <span class="keyword">default</span>:</span><br><span class="line">                    <span class="keyword">break</span>;</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="comment">//关键字段更新才执行后续操作</span></span><br><span class="line">        <span class="keyword">if</span> (update) &#123;</span><br><span class="line">            log.info(<span class="string">&quot;sync member:&#123;&#125;&quot;</span>, m);</span><br><span class="line">            <span class="keyword">if</span> (eventType == EventType.DELETE || Objects.equals(m.getIsDeleted(), <span class="string">&quot;1&quot;</span>)) &#123;</span><br><span class="line">                deleteById(m.getId());</span><br><span class="line">            &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">                saveOrUpdateIndex(m);</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;</span><br><span class="line">    </span><br><span class="line">    &#125;</span><br><span class="line">    </span><br><span class="line">    <span class="function"><span class="keyword">private</span> <span class="keyword">boolean</span> <span class="title">checkUpdate</span><span class="params">(<span class="keyword">boolean</span> u, <span class="keyword">boolean</span> updated)</span> </span>&#123;</span><br><span class="line">        <span class="keyword">if</span> (u || updated) &#123;</span><br><span class="line">            <span class="keyword">return</span> <span class="keyword">true</span>;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="keyword">return</span> <span class="keyword">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    </span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">saveOrUpdateIndex</span><span class="params">(Member member)</span> </span>&#123;</span><br><span class="line">        XContentBuilder builder;</span><br><span class="line">        <span class="keyword">try</span> &#123;</span><br><span class="line">            builder = XContentFactory.jsonBuilder()</span><br><span class="line">                    .startObject()</span><br><span class="line">                    .field(<span class="string">&quot;id&quot;</span>, member.getId())</span><br><span class="line">                    .field(<span class="string">&quot;mobile&quot;</span>, member.getMobile())</span><br><span class="line">                    .field(<span class="string">&quot;name&quot;</span>, member.getName())</span><br><span class="line">                    .field(<span class="string">&quot;py&quot;</span>, member.getPy())</span><br><span class="line">                    .endObject();</span><br><span class="line">        &#125; <span class="keyword">catch</span> (Exception e) &#123;</span><br><span class="line">            e.printStackTrace();</span><br><span class="line">            <span class="keyword">return</span>;</span><br><span class="line">        &#125;</span><br><span class="line">    </span><br><span class="line">        GetResponse getResponse = searchById(member.getId());</span><br><span class="line">        <span class="keyword">if</span> (<span class="keyword">null</span> == getResponse || <span class="keyword">null</span> == getResponse.getSourceAsString()) &#123;</span><br><span class="line">            <span class="comment">//创建索引</span></span><br><span class="line">            IndexResponse indexResponse = transportClient.prepareIndex(index, type, member.getId()).setSource(builder).execute().actionGet();</span><br><span class="line">            System.out.println(indexResponse);</span><br><span class="line">    </span><br><span class="line">        &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">            <span class="comment">//更新索引</span></span><br><span class="line">            <span class="keyword">try</span> &#123;</span><br><span class="line">                UpdateResponse response = transportClient.prepareUpdate(index, type, member.getId()).setDoc(builder).get();</span><br><span class="line">                System.out.println(response);</span><br><span class="line">            &#125; <span class="keyword">catch</span> (Exception e) &#123;</span><br><span class="line">                e.printStackTrace();</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">public</span> GetResponse <span class="title">searchById</span><span class="params">(String id)</span> </span>&#123;</span><br><span class="line">        GetResponse response = transportClient.prepareGet(index, type, id).execute().actionGet();</span><br><span class="line">        <span class="keyword">return</span> response;</span><br><span class="line">    &#125;</span><br><span class="line">    </span><br><span class="line">    <span class="function"><span class="keyword">public</span> DeleteResponse <span class="title">deleteById</span><span class="params">(String id)</span> </span>&#123;</span><br><span class="line">        DeleteResponse response = transportClient.prepareDelete(index, type, id).execute().actionGet();</span><br><span class="line">        log.info(<span class="string">&quot;deleteById:&#123;&#125;,response:&#123;&#125;&quot;</span>, id, response);</span><br><span class="line">        <span class="keyword">return</span> response;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>

<p>Member.java</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">Member</span> <span class="keyword">implements</span> <span class="title">Serializable</span> </span>&#123;</span><br><span class="line">    <span class="keyword">private</span> String id;</span><br><span class="line">    <span class="keyword">private</span> String mobile;</span><br><span class="line">    <span class="keyword">private</span> String name;</span><br><span class="line">    <span class="keyword">private</span> String py;</span><br><span class="line">    <span class="keyword">private</span> String isDeleted;</span><br><span class="line"> &#125;</span><br></pre></td></tr></table></figure>

<p>canal client使用springboot启动，此处不展开</p>
<p>启动clinet</p>
<h4 id="三、logstash的处理"><a href="#三、logstash的处理" class="headerlink" title="三、logstash的处理"></a>三、logstash的处理</h4><p>canal只能处理增量的数据，那么历史的怎么办呢，这个地方使用logstash导入，配置文件如下</p>
<p>sync.conf</p>
<figure class="highlight properties"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">input</span> <span class="string">&#123;</span></span><br><span class="line">    <span class="attr">stdin</span> <span class="string">&#123;</span></span><br><span class="line">    <span class="attr">&#125;</span></span><br><span class="line">    <span class="attr">jdbc</span> <span class="string">&#123;</span></span><br><span class="line"><span class="comment">      # mysql 数据库链接</span></span><br><span class="line">      <span class="attr">jdbc_connection_string</span> =<span class="string">&gt; &quot;jdbc:mysql://数据库连接&quot;</span></span><br><span class="line"><span class="comment">      # 用户名和密码</span></span><br><span class="line">      <span class="attr">jdbc_user</span> =<span class="string">&gt; &quot;数据库用户名&quot;</span></span><br><span class="line">      <span class="attr">jdbc_password</span> =<span class="string">&gt; &quot;数据库密码&quot;</span></span><br><span class="line"><span class="comment">      # 驱动</span></span><br><span class="line">      <span class="attr">jdbc_driver_library</span> =<span class="string">&gt; &quot;/usr/local/logstash/mysqldoc/mysql-connector-java-8.0.8-dmr.jar&quot;</span></span><br><span class="line"><span class="comment">      # 驱动类名</span></span><br><span class="line">      <span class="attr">jdbc_driver_class</span> =<span class="string">&gt; &quot;com.mysql.jdbc.Driver&quot;</span></span><br><span class="line">      <span class="attr">jdbc_paging_enabled</span> =<span class="string">&gt; &quot;true&quot;</span></span><br><span class="line">      <span class="attr">jdbc_page_size</span> =<span class="string">&gt; &quot;50000&quot;</span></span><br><span class="line"><span class="comment">      # 执行的sql</span></span><br><span class="line">      <span class="attr">statement</span> =<span class="string">&gt; &quot;SELECT m.id as id,m.pinyin_code as py,m.name,m.mobile FROM 用户表 m WHERE length(m.name) &gt; 1 and is_deleted=0   ORDER BY m.create_time DESC&quot;</span></span><br><span class="line">      <span class="attr">type</span> =<span class="string">&gt; &quot;es创建的type&quot;</span></span><br><span class="line"> <span class="attr">&#125;</span></span><br><span class="line"><span class="attr">&#125;</span></span><br><span class="line"></span><br><span class="line"><span class="attr">filter</span> <span class="string">&#123;</span></span><br><span class="line">    <span class="attr">json</span> <span class="string">&#123;</span></span><br><span class="line">        <span class="attr">source</span> =<span class="string">&gt; &quot;message&quot;</span></span><br><span class="line">        <span class="attr">remove_field</span> =<span class="string">&gt; [&quot;message&quot;]</span></span><br><span class="line">    <span class="attr">&#125;</span></span><br><span class="line"><span class="attr">&#125;</span></span><br><span class="line"></span><br><span class="line"><span class="attr">output</span> <span class="string">&#123;</span></span><br><span class="line">    <span class="attr">elasticsearch</span> <span class="string">&#123;</span></span><br><span class="line">        <span class="attr">hosts</span> =<span class="string">&gt; [&quot;es服务器&quot;]</span></span><br><span class="line">        <span class="attr">index</span> =<span class="string">&gt; &quot;创建的所以&quot;</span></span><br><span class="line">        <span class="attr">user</span>  =<span class="string">&gt; &quot;xpack用户名&quot;</span></span><br><span class="line">        <span class="attr">password</span> =<span class="string">&gt; &quot;xpack密码&quot;</span></span><br><span class="line">        <span class="attr">document_id</span> =<span class="string">&gt; &quot;%&#123;id&#125;&quot;</span></span><br><span class="line">    <span class="attr">&#125;</span></span><br><span class="line">    <span class="attr">stdout</span> <span class="string">&#123;</span></span><br><span class="line">        <span class="attr">codec</span> =<span class="string">&gt; json_lines</span></span><br><span class="line">    <span class="attr">&#125;</span></span><br><span class="line"><span class="attr">&#125;</span></span><br></pre></td></tr></table></figure>

<p>启动logstatsh同步</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">bin/logstash -f sync.conf</span><br></pre></td></tr></table></figure>

<h4 id="四、查询代码"><a href="#四、查询代码" class="headerlink" title="四、查询代码"></a>四、查询代码</h4><p>从es中查询数据</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="function"><span class="keyword">public</span> List&lt;Member&gt; <span class="title">queryMember</span><span class="params">(String keyWord)</span> </span>&#123;</span><br><span class="line">        <span class="keyword">long</span> start = System.currentTimeMillis();</span><br><span class="line">        <span class="comment">//搜索用户</span></span><br><span class="line">        List&lt;Member&gt; elasticList = <span class="keyword">new</span> ArrayList&lt;&gt;(<span class="number">30</span>);</span><br><span class="line">        <span class="keyword">if</span>(StringUtils.isBlank(keyWord))&#123;</span><br><span class="line">            <span class="keyword">return</span> elasticList;</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        <span class="comment">//ES检索并排序</span></span><br><span class="line">        <span class="keyword">try</span> &#123;</span><br><span class="line"></span><br><span class="line">            BoolQueryBuilder booleanQuery = QueryBuilders.boolQuery();</span><br><span class="line"></span><br><span class="line">            QueryBuilder match = QueryBuilders.queryStringQuery(keyWord)</span><br><span class="line">                    .field(<span class="string">&quot;name&quot;</span>).field(<span class="string">&quot;mobile&quot;</span>).field(<span class="string">&quot;py&quot;</span>).defaultOperator(Operator.AND).boost(<span class="number">100</span>);</span><br><span class="line">            booleanQuery.should(match);</span><br><span class="line"></span><br><span class="line">            QueryBuilder like =  QueryBuilders.queryStringQuery(keyWord+<span class="string">&quot;*&quot;</span>)</span><br><span class="line">                    .field(<span class="string">&quot;name&quot;</span>).field(<span class="string">&quot;mobile&quot;</span>).field(<span class="string">&quot;py&quot;</span>).defaultOperator(Operator.AND).boost(<span class="number">50</span>);</span><br><span class="line">            booleanQuery.should(like);</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">            QueryBuilder likeAll =  QueryBuilders.queryStringQuery(<span class="string">&quot;*&quot;</span>+keyWord+<span class="string">&quot;*&quot;</span>)</span><br><span class="line">                    .field(<span class="string">&quot;name&quot;</span>).field(<span class="string">&quot;mobile&quot;</span>).field(<span class="string">&quot;py&quot;</span>).defaultOperator(Operator.AND).boost(<span class="number">50</span>);</span><br><span class="line">            booleanQuery.should(likeAll);</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line">            SearchHits searchHits = transportClient.prepareSearch(memberIndex)</span><br><span class="line">                    .setSearchType(SearchType.QUERY_THEN_FETCH)</span><br><span class="line">                    .setTimeout(TimeValue.timeValueSeconds(<span class="number">3</span>)).</span><br><span class="line">                            setExplain(<span class="keyword">true</span>).setQuery(booleanQuery).setFrom(<span class="number">0</span>).setSize(<span class="number">30</span>).get()</span><br><span class="line">                    .getHits();</span><br><span class="line">            searchHits.forEach(searchHitFields -&gt; &#123;</span><br><span class="line">                Map&lt;String, Object&gt; map = searchHitFields.getSource();</span><br><span class="line">                String name = (String) map.get(<span class="string">&quot;name&quot;</span>);</span><br><span class="line">                String mobile = (String) map.get(<span class="string">&quot;mobile&quot;</span>);</span><br><span class="line">                String py = (String) map.get(<span class="string">&quot;py&quot;</span>);</span><br><span class="line">                String id = (String) map.get(<span class="string">&quot;id&quot;</span>);</span><br><span class="line">                Member m = <span class="keyword">new</span> Member();</span><br><span class="line">                m.setId(id);</span><br><span class="line">                m.setName(name);</span><br><span class="line">                m.setMobile(mobile);</span><br><span class="line">                m.setPy(py);</span><br><span class="line">                elasticList.add(m);</span><br><span class="line">            &#125;);</span><br><span class="line">        &#125; <span class="keyword">catch</span> (Exception e) &#123;</span><br><span class="line">            log.error(<span class="string">&quot;es搜索异常:&#123;&#125;&quot;</span>, ExceptionStackUtil.getExceptionStackTrace(e));</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="keyword">long</span> end = System.currentTimeMillis();</span><br><span class="line">        log.info(<span class="string">&quot;从es中搜索出来的值 key:&#123;&#125; time:&#123;&#125;ms res：&#123;&#125;&quot;</span>,keyWord, end - start, elasticList.size());</span><br><span class="line">        <span class="keyword">return</span> elasticList;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>

      
    </div>
    <footer class="article-footer">
      <a data-url="https://shouzimu.gitee.io/2020/11/12/essync/" data-id="ckhesflrz0007zoiz1comh6sf" class="article-share-link">Share</a>
      
      
  <ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/elasticsearch/" rel="tag">elasticsearch</a></li></ul>

    </footer>
  </div>
  
</article>


  
    <article id="post-centos-static-ip" class="article article-type-post" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2020/11/12/centos-static-ip/" class="article-date">
  <time datetime="2020-11-12T12:02:57.000Z" itemprop="datePublished">2020-11-12</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 itemprop="name">
      <a class="article-title" href="/2020/11/12/centos-static-ip/">centos配置静态ip</a>
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <p>此处环境为vmwork station 15+centos7，网络使用nat的方式</p>
<p>目标ip: 192.168.1.131，当前ip192.168.31.133</p>
<p>先查看ip地址</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">[root@localhost ~]# ip addr</span><br><span class="line">.</span><br><span class="line">.</span><br><span class="line">2: ens33: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu 1500 qdisc pfifo_fast state UP group default qlen 1000</span><br><span class="line">    inet 192.168.31.133/24 brd 192.168.31.255 scope global noprefixroute dynamic ens33</span><br><span class="line">.</span><br><span class="line">.</span><br></pre></td></tr></table></figure>

<p>当前的ip为192.168.31.133，对应使用的网络为ens33</p>
<p>编辑network-scripts下对应的文件</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line">vi /etc/sysconfig/network-scripts/ifcfg-ens33 </span><br><span class="line">TYPE=&quot;Ethernet&quot;</span><br><span class="line">PROXY_METHOD=&quot;none&quot;</span><br><span class="line">BROWSER_ONLY=&quot;no&quot;</span><br><span class="line">BOOTPROTO=&quot;dhcp&quot;</span><br><span class="line">DEFROUTE=&quot;yes&quot;</span><br><span class="line">IPV4_FAILURE_FATAL=&quot;no&quot;</span><br><span class="line">IPV6INIT=&quot;yes&quot;</span><br><span class="line">IPV6_AUTOCONF=&quot;yes&quot;</span><br><span class="line">IPV6_DEFROUTE=&quot;yes&quot;</span><br><span class="line">IPV6_FAILURE_FATAL=&quot;no&quot;</span><br><span class="line">IPV6_ADDR_GEN_MODE=&quot;stable-privacy&quot;</span><br><span class="line">NAME=&quot;ens33&quot;</span><br><span class="line">DEVICE=&quot;ens33&quot;</span><br><span class="line">ONBOOT=&quot;yes&quot;</span><br><span class="line"></span><br><span class="line">修改为其中几项如下</span><br><span class="line">TYPE=&quot;Ethernet&quot;</span><br><span class="line">BOOTPROTO=&quot;static&quot;</span><br><span class="line">IPADDR=192.168.31.131</span><br><span class="line">GATEWAY=192.168.31.2</span><br><span class="line">DNS1=DNS1</span><br><span class="line">DNS2=DNS2</span><br><span class="line">NETMASK=255.255.255.0</span><br></pre></td></tr></table></figure>



<p>重启网络就可以看到新的ip了</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">[root@localhost ~]# service network restart</span><br><span class="line">Restarting network (via systemctl):                        [  确定  ]</span><br><span class="line">[root@localhost ~]# ip addr</span><br><span class="line">.</span><br><span class="line">.</span><br><span class="line">2: ens33: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu 1500 qdisc pfifo_fast state UP group default qlen 1000</span><br><span class="line">.</span><br><span class="line">.</span><br><span class="line">inet 192.168.31.131&#x2F;24 brd 192.168.31.255 scope global noprefixroute ens33</span><br></pre></td></tr></table></figure>
      
    </div>
    <footer class="article-footer">
      <a data-url="https://shouzimu.gitee.io/2020/11/12/centos-static-ip/" data-id="ckhes9fyy0006zoiz38zb6gcj" class="article-share-link">Share</a>
      
      
  <ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/os/" rel="tag">os</a></li></ul>

    </footer>
  </div>
  
</article>


  
    <article id="post-remote-debug" class="article article-type-post" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2020/11/12/remote-debug/" class="article-date">
  <time datetime="2020-11-12T12:01:54.000Z" itemprop="datePublished">2020-11-12</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 itemprop="name">
      <a class="article-title" href="/2020/11/12/remote-debug/">idea远程调试</a>
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <p>此处以jdbcT4-2.2.0作为测试例子</p>
<h4 id="一、原理"><a href="#一、原理" class="headerlink" title="一、原理"></a>一、原理</h4><p>远程调试的原理就是两个VM之间通过debug协议进行通信，两者之间通过socket进行通信。</p>
<h4 id="二、实现"><a href="#二、实现" class="headerlink" title="二、实现"></a>二、实现</h4><p>在启动JVM时加上参数</p>
<p>如：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">-agentlib:jdwp&#x3D;transport&#x3D;dt_socket,address&#x3D;8001,server&#x3D;y,suspend&#x3D;n</span><br></pre></td></tr></table></figure>

<p>参数说明:</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">- transport :调试程序和VM之间的通信方式， dt_socket 表示用套接字传输</span><br><span class="line">- server&#x3D;y : 表示是监听其他debugclient端的请求</span><br><span class="line">- suspend : 表示是否在调试客户端建立连接之后启动 VM。</span><br><span class="line">+ 如果为y，那么当前的VM就是suspend直到有debug client连接进来才开始执行程序</span><br><span class="line">+ 如果为n，那么当前的VM就会直接执行，不会等待debug client连接进来</span><br><span class="line">- address&#x3D;8001 : 表示端口是5005</span><br></pre></td></tr></table></figure>



<h4 id="三、运行环境"><a href="#三、运行环境" class="headerlink" title="三、运行环境"></a>三、运行环境</h4><p>环境如下：</p>
<table>
<thead>
<tr>
<th>软件</th>
<th>位置</th>
<th>版本</th>
</tr>
</thead>
<tbody><tr>
<td>虚拟机ip</td>
<td>192.168.31.130</td>
<td></td>
</tr>
<tr>
<td>操作系统</td>
<td>虚拟机</td>
<td>CentOS Linux release 7.7.1908 (Core)</td>
</tr>
<tr>
<td>trafodion</td>
<td>虚拟机k8s中</td>
<td>2.7.0</td>
</tr>
<tr>
<td>服务端java</td>
<td>虚拟机</td>
<td>openjdk version “1.8.0_242”</td>
</tr>
<tr>
<td>idea</td>
<td>主机windows</td>
<td>2020.1</td>
</tr>
<tr>
<td>java</td>
<td>主机windows</td>
<td>java version “1.8.0_251”</td>
</tr>
<tr>
<td>hbase</td>
<td>虚拟机</td>
<td>2.2.5</td>
</tr>
<tr>
<td>hadoop</td>
<td>虚拟机</td>
<td>2.10.0</td>
</tr>
</tbody></table>
<h4 id="四、服务端准备工作"><a href="#四、服务端准备工作" class="headerlink" title="四、服务端准备工作"></a>四、服务端准备工作</h4><p>测试sql:</p>
<figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">create</span> <span class="keyword">table</span> test001(<span class="keyword">id</span> <span class="built_in">int</span> ,primary <span class="keyword">key</span>(<span class="keyword">id</span>));</span><br><span class="line"><span class="keyword">insert</span> <span class="keyword">into</span> test001(<span class="keyword">id</span>) <span class="keyword">values</span>(<span class="number">1</span>),(<span class="number">2</span>),(<span class="number">3</span>),(<span class="number">4</span>),(<span class="number">5</span>),(<span class="number">6</span>),(<span class="number">7</span>);</span><br></pre></td></tr></table></figure>

<p>远程运行的测试代码如下 <code>Test.java</code></p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> java.sql.*;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">Test</span> </span>&#123;</span><br><span class="line">    <span class="keyword">protected</span> <span class="keyword">static</span> <span class="keyword">final</span> String url = <span class="string">&quot;jdbc:t4jdbc://10.103.0.125:23400/:&quot;</span>;</span><br><span class="line">    <span class="keyword">protected</span> <span class="keyword">static</span> <span class="keyword">final</span> String driverClass = <span class="string">&quot;org.trafodion.jdbc.t4.T4Driver&quot;</span>;</span><br><span class="line">    <span class="keyword">protected</span> <span class="keyword">static</span> <span class="keyword">final</span> String userName = <span class="string">&quot;DB__ROOT&quot;</span>;</span><br><span class="line">    <span class="keyword">protected</span> <span class="keyword">static</span> <span class="keyword">final</span> String pwd = <span class="string">&quot;traf123&quot;</span>;</span><br><span class="line">    <span class="keyword">protected</span> <span class="keyword">static</span> Connection conn;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">main</span><span class="params">(String[] args)</span> <span class="keyword">throws</span> Exception </span>&#123;</span><br><span class="line">        System.out.println(<span class="string">&quot;enter word continue..&quot;</span>);</span><br><span class="line">        System.in.read();<span class="comment">//阻塞远程程序，client端有时间连接</span></span><br><span class="line">        System.out.println(<span class="string">&quot;exe..&quot;</span>);</span><br><span class="line">        Class.forName(driverClass);</span><br><span class="line">        conn = DriverManager.getConnection(url, userName, pwd);</span><br><span class="line">        String sql = <span class="string">&quot;select * from test001 order by id desc&quot;</span>;</span><br><span class="line">        String countSql = <span class="string">&quot;select count(*) from test001 order by id desc&quot;</span>;</span><br><span class="line">        Statement ps = conn.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_READ_ONLY);</span><br><span class="line">        Statement psCount = conn.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_READ_ONLY);</span><br><span class="line">        ResultSet rs = ps.executeQuery(sql);</span><br><span class="line">        ResultSet countRs = psCount.executeQuery(countSql);</span><br><span class="line">        <span class="keyword">if</span>(countRs.next())&#123;</span><br><span class="line">            System.out.println(<span class="string">&quot;count:&quot;</span> + countRs.getInt(<span class="number">1</span>));</span><br><span class="line">        &#125;<span class="keyword">else</span>&#123;</span><br><span class="line">            System.out.println(<span class="string">&quot;count null&quot;</span>);</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="keyword">while</span> (rs.next()) &#123;</span><br><span class="line">            System.out.println(rs.getInt(<span class="number">1</span>));</span><br><span class="line">        &#125;</span><br><span class="line">        ps.close();</span><br><span class="line">        conn.close();</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>



<p>程序运行脚本如下：</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">//将驱动包添加到classpath中</span><br><span class="line">export CLASSPATH=/root/test/jdbcT4-2.2.0.jar:.</span><br><span class="line">//编译</span><br><span class="line">javac Test.java</span><br><span class="line">//运行</span><br><span class="line">java -agentlib:jdwp=transport=dt_socket,address=8001,server=y,suspend=n Test</span><br><span class="line"></span><br><span class="line">会看到如下输出</span><br><span class="line">Listening for transport dt_socket at address: 8001</span><br><span class="line">enter word continue..</span><br></pre></td></tr></table></figure>

<p>运行Test后，会等待输入才会继续运行，下面处理idea中相关的工作</p>
<h4 id="五、idea中的操作"><a href="#五、idea中的操作" class="headerlink" title="五、idea中的操作"></a>五、idea中的操作</h4><h5 id="1、项目准备"><a href="#1、项目准备" class="headerlink" title="1、项目准备"></a>1、项目准备</h5><p>此处以maven项目为例，只是为了运行的时候代码跟远程的能关联上</p>
<p>idea中新建一个maven项目为此处为<code>remote-debugger</code></p>
<p><code>pom.xml</code></p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">&lt;dependency&gt;</span><br><span class="line">    &lt;groupId&gt;org.apache.trafodion.jdbc.t4&lt;&#x2F;groupId&gt;</span><br><span class="line">    &lt;artifactId&gt;jdbcT4&lt;&#x2F;artifactId&gt;</span><br><span class="line">    &lt;version&gt;2.2.0&lt;&#x2F;version&gt;</span><br><span class="line">&lt;&#x2F;dependency&gt;</span><br></pre></td></tr></table></figure>

<p>选择2.2.0的原因是从maven中央仓库上可以直接下载到源码</p>
<h5 id=""><a href="#" class="headerlink" title=""></a></h5><h5 id="2、添加remote运行程序"><a href="#2、添加remote运行程序" class="headerlink" title="2、添加remote运行程序"></a>2、添加remote运行程序</h5><p>点击上方运行框添加一个remote的配置</p>
<p><img src="https://modprobe.oss-cn-beijing.aliyuncs.com/esgyn/01-add-run.png" alt="01-add-run"></p>
<p>配置对应属性</p>
<p>host为远程程序所在的ip地址，port填写远程程序对应address参数，use module classpath选择源码能和远程对应上的项目，点击apply就可以了</p>
<p><img src="https://modprobe.oss-cn-beijing.aliyuncs.com/esgyn/02-add-remote.png"></p>
<h5 id="3、开始debugger"><a href="#3、开始debugger" class="headerlink" title="3、开始debugger"></a>3、开始debugger</h5><p>确定远程jvm处于运行状态后以<code>debug的方式</code>运行刚刚创建的remote configurations，就是图中的虫子图标，成功</p>
<p>后下面控制台有对应的输出。</p>
<p><img src="https://modprobe.oss-cn-beijing.aliyuncs.com/esgyn/03-connect.png"></p>
<p>在图中111行左键单击添加一个断点，然后到远程任意输入字符让程序继续运行可以看到程序运行到了这个位置</p>
<p>右键属性可以查看或者编辑变量，下方debugger窗口也可以查看。</p>
<p><img src="https://modprobe.oss-cn-beijing.aliyuncs.com/esgyn/03-get-change.png"></p>
<p>编辑变量，将user的值由原来的DB__ROOT修改为hello world，确定后继续</p>
<p><img src="https://modprobe.oss-cn-beijing.aliyuncs.com/esgyn/04-change.png"></p>
<p>修改变量后可以看到user改变后报错了，可以使用这种方式修改运行时的值</p>
<p><img src="https://modprobe.oss-cn-beijing.aliyuncs.com/esgyn/05-remote-error.png"></p>
<h4 id="六、Hbase-debug"><a href="#六、Hbase-debug" class="headerlink" title="六、Hbase debug"></a>六、Hbase debug</h4><h5 id="1、hbase参数配置"><a href="#1、hbase参数配置" class="headerlink" title="1、hbase参数配置"></a>1、hbase参数配置</h5><p>首先设置hbase启动脚本</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">vim conf/hbase-env.sh </span><br><span class="line"></span><br><span class="line">添加内容</span><br><span class="line">export HBASE_MASTER_OPTS=&quot;$HBASE_MASTER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=10445&quot;</span><br><span class="line">export HBASE_REGIONSERVER_OPTS=&quot;$HBASE_REGIONSERVER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=10446&quot;</span><br><span class="line"></span><br><span class="line">设置了master的调试端口为10445，regions的为10446</span><br><span class="line"></span><br><span class="line">启动hbase</span><br><span class="line">sh bin/start-hbase.sh</span><br><span class="line">输出...</span><br><span class="line">Listening for transport dt_socket at address: 10445</span><br></pre></td></tr></table></figure>

<h5 id="2、idea调试设置"><a href="#2、idea调试设置" class="headerlink" title="2、idea调试设置"></a>2、idea调试设置</h5><p>在maven中添加依赖</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">&lt;dependency&gt;</span><br><span class="line">    &lt;groupId&gt;org.apache.hbase&lt;&#x2F;groupId&gt;</span><br><span class="line">    &lt;artifactId&gt;hbase-server&lt;&#x2F;artifactId&gt;</span><br><span class="line">    &lt;version&gt;2.2.5&lt;&#x2F;version&gt;</span><br><span class="line">&lt;&#x2F;dependency&gt;</span><br></pre></td></tr></table></figure>

<p>添加一个remote程序，如果调试master相关代码则使用上面配置的<code>10445 </code>端口</p>
<p><img src="https://modprobe.oss-cn-beijing.aliyuncs.com/esgyn/07-remote-hb.png"></p>
<h5 id="3、调试建表代码"><a href="#3、调试建表代码" class="headerlink" title="3、调试建表代码"></a>3、调试建表代码</h5><p>建表相关的代码在<code>HMaster#createTable</code>中，打一个断点</p>
<p>然后到服务器(虚拟机)上启动shell进行测试</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"> ./bin/hbase shell</span><br><span class="line"></span><br><span class="line">hbase(main):003:0&gt; create &#x27;test&#x27;, &#x27;cf&#x27;</span><br><span class="line">ERROR: Table already exists: test!</span><br><span class="line">For usage try &#x27;help &quot;create&quot;&#x27;</span><br><span class="line">Took 677.1992 seconds  </span><br></pre></td></tr></table></figure>

<p>可以看到程序停在了我们设置的断点位置</p>
<p><img src="https://modprobe.oss-cn-beijing.aliyuncs.com/esgyn/08-hmaster-01.png"></p>
<h4 id="七、idea调试技巧"><a href="#七、idea调试技巧" class="headerlink" title="七、idea调试技巧"></a>七、idea调试技巧</h4><p>可以参考博客 <a target="_blank" rel="noopener" href="https://www.cnblogs.com/diaobiyong/p/10682996.html">在Intellij IDEA中使用Debug</a></p>

      
    </div>
    <footer class="article-footer">
      <a data-url="https://shouzimu.gitee.io/2020/11/12/remote-debug/" data-id="ckhes83al0005zoiz8a4cfrj8" class="article-share-link">Share</a>
      
      
  <ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/java/" rel="tag">java</a></li></ul>

    </footer>
  </div>
  
</article>


  
    <article id="post-k8s-single" class="article article-type-post" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2020/11/12/k8s-single/" class="article-date">
  <time datetime="2020-11-12T12:00:26.000Z" itemprop="datePublished">2020-11-12</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 itemprop="name">
      <a class="article-title" href="/2020/11/12/k8s-single/">Centos7安装单节点kubernetes</a>
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <p>官方教程链接：<a target="_blank" rel="noopener" href="https://kubernetes.io/zh/docs/tasks/tools/install-kubectl/">https://kubernetes.io/zh/docs/tasks/tools/install-kubectl/</a></p>
<h2 id="安装-kubectl"><a href="#安装-kubectl" class="headerlink" title="安装 kubectl"></a>安装 kubectl</h2><p>1、配置使用阿里云的源</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">cat &lt;&lt;EOF &gt; /etc/yum.repos.d/kubernetes.repo</span><br><span class="line">[kubernetes]</span><br><span class="line">name=Kubernetes</span><br><span class="line">baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64</span><br><span class="line">enabled=1</span><br><span class="line">gpgcheck=1</span><br><span class="line">repo_gpgcheck=1</span><br><span class="line">gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg</span><br><span class="line">EOF</span><br></pre></td></tr></table></figure>



<p>2、安装kubectl</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum install -y kubectl</span><br></pre></td></tr></table></figure>



<h3 id="安装-Minikube"><a href="#安装-Minikube" class="headerlink" title="安装 Minikube"></a>安装 Minikube</h3><p>1、下载minikube</p>
<p>可以下载并使用一个单节点二进制文件。</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 \</span><br><span class="line">  &amp;&amp; chmod +x minikube</span><br></pre></td></tr></table></figure>

<p>不过由于googleapis不能访问，我们使用如下命令从github上下载，具体版本号可以到github上查看</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">curl -Lo minikube https:&#x2F;&#x2F;github.com&#x2F;kubernetes&#x2F;minikube&#x2F;releases&#x2F;download&#x2F;v1.10.0-beta.1&#x2F;minikube-linux-amd64   &amp;&amp; chmod +x minikube</span><br></pre></td></tr></table></figure>



<p>2、将 Minikube 可执行文件添加至 path</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">sudo mkdir -p /usr/local/bin/</span><br><span class="line">sudo install minikube /usr/local/bin/</span><br></pre></td></tr></table></figure>



<p>3、启动</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">[root@iZm5e7y9qoyl006b96mwaaZ k8s]#  minikube start --driver=none</span><br><span class="line">* minikube v1.10.0-beta.1 on Centos 7.7.1908</span><br><span class="line">* Using the none driver based on user configuration</span><br><span class="line">X Sorry, Kubernetes v1.18.0 requires conntrack to be installed in root&#x27;s path</span><br></pre></td></tr></table></figure>

<p>发现报了一个错误</p>
<p>安装conntrack</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum install -y conntrack-tools</span><br></pre></td></tr></table></figure>

<p>再次执行进入漫长的下载</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">[root@iZm5e7y9qoyl006b96mwaaZ k8s]#  minikube start --driver&#x3D;none</span><br><span class="line">* minikube v1.10.0-beta.1 on Centos 7.7.1908</span><br><span class="line">* Using the none driver based on user configuration</span><br><span class="line">* Starting control plane node minikube in cluster minikube</span><br><span class="line">* Running on localhost (CPUs&#x3D;2, Memory&#x3D;3789MB, Disk&#x3D;40183MB) ...</span><br><span class="line">* OS release is CentOS Linux 7 (Core)</span><br><span class="line">* Preparing Kubernetes v1.18.0 on Docker 1.13.1 ...</span><br><span class="line">    &gt; kubectl.sha256: 65 B &#x2F; 65 B [--------------------------] 100.00% ? p&#x2F;s 0s</span><br><span class="line">    &gt; kubelet.sha256: 65 B &#x2F; 65 B [--------------------------] 100.00% ? p&#x2F;s 0s</span><br><span class="line">! This bare metal machine is having trouble accessing https:&#x2F;&#x2F;k8s.gcr.io</span><br><span class="line">* To pull new external images, you may need to configure a proxy: https:&#x2F;&#x2F;minikube.sigs.k8s.io&#x2F;docs&#x2F;reference&#x2F;networking&#x2F;proxy&#x2F;</span><br><span class="line">    &gt; kubeadm.sha256: 65 B &#x2F; 65 B [--------------------------] 100.00% ? p&#x2F;s 0s</span><br><span class="line">    &gt; kubectl: 41.98 MiB &#x2F; 41.98 MiB [---------------] 100.00% 11.29 MiB p&#x2F;s 4s</span><br><span class="line">    &gt; kubeadm: 37.96 MiB &#x2F; 37.96 MiB [----------------] 100.00% 7.99 MiB p&#x2F;s 5s</span><br><span class="line">    &gt; kubelet: 2.35 MiB &#x2F; 108.01 MiB [&gt;_______] 2.17% 11.00 KiB p&#x2F;s ETA 2h44m0s</span><br></pre></td></tr></table></figure>

<p>可以在启动命令上追加</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">--registry-mirror=https://替换为阿里云加速.mirror.aliyuncs.com</span><br><span class="line">minikube start --driver=none --image-repository=&#x27;registry.aliyuncs.com/google_containers&#x27;</span><br></pre></td></tr></table></figure>

<p>经过漫长等待出现下面输出</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">! kubectl and minikube configuration will be stored in /root</span><br><span class="line">! To use kubectl or minikube commands as your own user, you may need to relocate them. For example, to overwrite your own settings, run:</span><br><span class="line">* </span><br><span class="line">  - sudo mv /root/.kube /root/.minikube $HOME</span><br><span class="line">  - sudo chown -R $USER $HOME/.kube $HOME/.minikube</span><br><span class="line">* </span><br><span class="line">* This can also be done automatically by setting the env var CHANGE_MINIKUBE_NONE_USER=true</span><br><span class="line">* Enabled addons: default-storageclass, storage-provisioner</span><br><span class="line">* Done! kubectl is now configured to use &quot;minikube&quot;</span><br></pre></td></tr></table></figure>

<p>说明启动成功</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">[root@iZm5e7y9qoyl006b96mwaaZ k8s]# minikube status</span><br><span class="line">minikube</span><br><span class="line">type: Control Plane</span><br><span class="line">host: Running</span><br><span class="line">kubelet: Running</span><br><span class="line">apiserver: Running</span><br><span class="line">kubeconfig: Configured</span><br><span class="line"></span><br><span class="line">[root@iZm5e7y9qoyl006b96mwaaZ k8s]# kubectl get nodes</span><br><span class="line">NAME                      STATUS   ROLES    AGE     VERSION</span><br><span class="line">izm5e7y9qoyl006b96mwaaz   Ready    master   3m22s   v1.18.0</span><br><span class="line"></span><br><span class="line">[root@iZm5e7y9qoyl006b96mwaaZ k8s]# kubectl  cluster-info</span><br><span class="line">Kubernetes master is running at https://172.31.202.153:8443</span><br><span class="line">KubeDNS is running at https://172.31.202.153:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy</span><br><span class="line"></span><br><span class="line">To further debug and diagnose cluster problems, use &#x27;kubectl cluster-info dump&#x27;.</span><br></pre></td></tr></table></figure>





<h3 id="测试"><a href="#测试" class="headerlink" title="测试"></a>测试</h3><p>官网链接：<a target="_blank" rel="noopener" href="https://kubernetes.io/docs/setup/learning-environment/minikube/">https://kubernetes.io/docs/setup/learning-environment/minikube/</a></p>
<p>1、使用名为 <code>echoserver</code> 的镜像创建一个 Kubernetes Deployment，并使用 <code>--port</code> 在端口 8080 上暴露服务。<code>echoserver</code> 是一个简单的 HTTP 服务器。</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">kubectl run hello-minikube --image=registry.aliyuncs.com/google_containers/echoserver:1.10 --port=8080</span><br><span class="line"></span><br><span class="line">上面这个网站描述输出的是pod，跟预期的deployment不符</span><br><span class="line">pod/hello-minikube created</span><br><span class="line"></span><br><span class="line">换成下面的命令试一下</span><br><span class="line">kubectl create deployment hello-minikube --image=registry.aliyuncs.com/google_containers/echoserver:1.10</span><br><span class="line">输出</span><br><span class="line">deployment.apps/hello-minikube created</span><br></pre></td></tr></table></figure>



<p>2、查看pod状态</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[root@iZm5e7y9qoyl006b96mwaaZ k8s]# kubectl get pod</span><br><span class="line">NAME             READY   STATUS    RESTARTS   AGE</span><br><span class="line">hello-minikube   1/1     Running   0          6m18s</span><br></pre></td></tr></table></figure>



<p>3、将<code>hello-minikube</code> Deployment，需要将其作为 Service 公开</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">kubectl expose deployment hello-minikube --type&#x3D;NodePort --port&#x3D;8080</span><br><span class="line"></span><br><span class="line">service&#x2F;hello-minikube exposed</span><br></pre></td></tr></table></figure>



<p>4、获取暴露 Service 的 URL 以查看 Service 的详细信息：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">minikube service hello-minikube --url</span><br></pre></td></tr></table></figure>


      
    </div>
    <footer class="article-footer">
      <a data-url="https://shouzimu.gitee.io/2020/11/12/k8s-single/" data-id="ckhes672e0004zoizfnthcadj" class="article-share-link">Share</a>
      
      
  <ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/cloud/" rel="tag">cloud</a></li></ul>

    </footer>
  </div>
  
</article>


  
    <article id="post-cdh-esgyn-change-ip" class="article article-type-post" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2020/11/12/cdh-esgyn-change-ip/" class="article-date">
  <time datetime="2020-11-12T11:06:28.000Z" itemprop="datePublished">2020-11-12</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 itemprop="name">
      <a class="article-title" href="/2020/11/12/cdh-esgyn-change-ip/">CDH+esgynDB 修改ip</a>
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <p>版本情况</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">cdh: 5.16.2</span><br><span class="line">esgyndb: 1.6.2</span><br></pre></td></tr></table></figure>

<p><strong><font color=#FF0000> 注：测试环境使用</font></strong></p>
<p>使用cdh在虚拟机中安装的esgyndb，后续由于切换ip，导致访问异常，此处将处理过程记录</p>
<p>1.查看postgresql的scm用户的密码</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">grep password /etc/cloudera-scm-server/db.properties</span><br></pre></td></tr></table></figure>



<p>2.登录postgresql数据库</p>
<figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">select</span> host_id, host_identifier, <span class="keyword">name</span>, ip_address <span class="keyword">from</span> <span class="keyword">hosts</span>;</span><br></pre></td></tr></table></figure>

<p><img src="https://modprobe.oss-cn-beijing.aliyuncs.com/gitee/cdh_pg.png"></p>
<p>3.修改postgresql数据库中的数据（即主机的ip）</p>
<figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">update</span> <span class="keyword">hosts</span> <span class="keyword">set</span> (ip_address) = (<span class="string">&#x27;new ip地址&#x27;</span>) <span class="keyword">where</span> host_id=<span class="string">&#x27;1&#x27;</span>;</span><br></pre></td></tr></table></figure>



<p>4.重启服务</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">service cloudera-scm-server-db restart</span><br><span class="line">service cloudera-scm-server restart</span><br><span class="line">service cloudera-scm-agent restart </span><br></pre></td></tr></table></figure>



<p>5.重启esgyndb后遇到一个问题，dcs返回的mxoserver仍然是老的ip，导致jdbc连接失败，处理如下</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">/opt/apache-zookeeper-3.5.8-bin/bin/zkCli.sh </span><br><span class="line"></span><br><span class="line">删除trafodion在zk上的注册信息</span><br><span class="line">rmr /trafodion  </span><br><span class="line"></span><br><span class="line">启动数据库</span><br><span class="line">sqstart</span><br></pre></td></tr></table></figure>


      
    </div>
    <footer class="article-footer">
      <a data-url="https://shouzimu.gitee.io/2020/11/12/cdh-esgyn-change-ip/" data-id="ckheq8soy0001zoize7ne6xi8" class="article-share-link">Share</a>
      
      
  <ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/esgynDB/" rel="tag">esgynDB</a></li></ul>

    </footer>
  </div>
  
</article>


  
    <article id="post-hello-world" class="article article-type-post" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2020/11/12/hello-world/" class="article-date">
  <time datetime="2020-11-12T10:46:26.554Z" itemprop="datePublished">2020-11-12</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 itemprop="name">
      <a class="article-title" href="/2020/11/12/hello-world/">Hello World</a>
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <p>Welcome to <a target="_blank" rel="noopener" href="https://hexo.io/">Hexo</a>! This is your very first post. Check <a target="_blank" rel="noopener" href="https://hexo.io/docs/">documentation</a> for more info. If you get any problems when using Hexo, you can find the answer in <a target="_blank" rel="noopener" href="https://hexo.io/docs/troubleshooting.html">troubleshooting</a> or you can ask me on <a target="_blank" rel="noopener" href="https://github.com/hexojs/hexo/issues">GitHub</a>.</p>
<h2 id="Quick-Start"><a href="#Quick-Start" class="headerlink" title="Quick Start"></a>Quick Start</h2><h3 id="Create-a-new-post"><a href="#Create-a-new-post" class="headerlink" title="Create a new post"></a>Create a new post</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo new <span class="string">&quot;My New Post&quot;</span></span><br></pre></td></tr></table></figure>

<p>More info: <a target="_blank" rel="noopener" href="https://hexo.io/docs/writing.html">Writing</a></p>
<h3 id="Run-server"><a href="#Run-server" class="headerlink" title="Run server"></a>Run server</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo server</span><br></pre></td></tr></table></figure>

<p>More info: <a target="_blank" rel="noopener" href="https://hexo.io/docs/server.html">Server</a></p>
<h3 id="Generate-static-files"><a href="#Generate-static-files" class="headerlink" title="Generate static files"></a>Generate static files</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo generate</span><br></pre></td></tr></table></figure>

<p>More info: <a target="_blank" rel="noopener" href="https://hexo.io/docs/generating.html">Generating</a></p>
<h3 id="Deploy-to-remote-sites"><a href="#Deploy-to-remote-sites" class="headerlink" title="Deploy to remote sites"></a>Deploy to remote sites</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo deploy</span><br></pre></td></tr></table></figure>

<p>More info: <a target="_blank" rel="noopener" href="https://hexo.io/docs/one-command-deployment.html">Deployment</a></p>

      
    </div>
    <footer class="article-footer">
      <a data-url="https://shouzimu.gitee.io/2020/11/12/hello-world/" data-id="ckhepk2fy0000zoizdymu2ug6" class="article-share-link">Share</a>
      
      
    </footer>
  </div>
  
</article>


  


</section>
        
          <aside id="sidebar">
  
    

  
    
  <div class="widget-wrap">
    <h3 class="widget-title">Tags</h3>
    <div class="widget">
      <ul class="tag-list" itemprop="keywords"><li class="tag-list-item"><a class="tag-list-link" href="/tags/cloud/" rel="tag">cloud</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/elasticsearch/" rel="tag">elasticsearch</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/esgynDB/" rel="tag">esgynDB</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/java/" rel="tag">java</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/os/" rel="tag">os</a></li></ul>
    </div>
  </div>


  
    
  <div class="widget-wrap">
    <h3 class="widget-title">Tag Cloud</h3>
    <div class="widget tagcloud">
      <a href="/tags/cloud/" style="font-size: 10px;">cloud</a> <a href="/tags/elasticsearch/" style="font-size: 10px;">elasticsearch</a> <a href="/tags/esgynDB/" style="font-size: 10px;">esgynDB</a> <a href="/tags/java/" style="font-size: 10px;">java</a> <a href="/tags/os/" style="font-size: 10px;">os</a>
    </div>
  </div>

  
    
  <div class="widget-wrap">
    <h3 class="widget-title">Archives</h3>
    <div class="widget">
      <ul class="archive-list"><li class="archive-list-item"><a class="archive-list-link" href="/archives/2020/11/">十一月 2020</a></li></ul>
    </div>
  </div>


  
    
  <div class="widget-wrap">
    <h3 class="widget-title">Recent Posts</h3>
    <div class="widget">
      <ul>
        
          <li>
            <a href="/2020/11/12/essync/">es+canal+logstash优化用户查询</a>
          </li>
        
          <li>
            <a href="/2020/11/12/centos-static-ip/">centos配置静态ip</a>
          </li>
        
          <li>
            <a href="/2020/11/12/remote-debug/">idea远程调试</a>
          </li>
        
          <li>
            <a href="/2020/11/12/k8s-single/">Centos7安装单节点kubernetes</a>
          </li>
        
          <li>
            <a href="/2020/11/12/cdh-esgyn-change-ip/">CDH+esgynDB 修改ip</a>
          </li>
        
      </ul>
    </div>
  </div>

  
</aside>
        
      </div>
      <footer id="footer">
  
  <div class="outer">
    <div id="footer-info" class="inner">
      &copy; 2020 shouzimu<br>
      Powered by <a href="http://hexo.io/" target="_blank">Hexo</a>
    </div>
  </div>
</footer>
    </div>
    <nav id="mobile-nav">
  
    <a href="/" class="mobile-nav-link">Home</a>
  
    <a href="/archives" class="mobile-nav-link">Archives</a>
  
</nav>
    

<script src="//ajax.googleapis.com/ajax/libs/jquery/2.0.3/jquery.min.js"></script>


  
<link rel="stylesheet" href="/fancybox/jquery.fancybox.css">

  
<script src="/fancybox/jquery.fancybox.pack.js"></script>




<script src="/js/script.js"></script>




  </div>
</body>
</html>